Improved Text Classification via Contrastive Adversarial Training

We propose a simple and general method to regularize the fine-tuning of Transformer-based encoders for text classification tasks. Specifically, during we generate adversarial examples by perturbing word embedding matrix model perform contrastive learning on clean in order teach learn noise-invariant representations. By training both along with additional objective, observe consistent improvement over standard examples. On several GLUE benchmark tasks, our fine-tuned Bert_Large outperforms baseline 1.7% average, Roberta_Large improves 1.3%. additionally validate different domains using three intent datasets, where 1-2% average. For challenging low-resource scenario, train system half data (per intent) each achieve similar performance compared trained full data.